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The Consortium for Student Retention Data Exchange: 

A Case Study of Consortium Development 

Abstract 

The Consortium for Student Retention Data Exchange was established in 1994. Its 
membership increased from 163 colleges and universities in 1994 to more than 400 in 2001. Over 
the past eight years, the routine of working with a large and diverse group of institutions has 
culminated in a valuable collection of experiences and lessons. The eight-year consortium history 
was documented in a case study of consortium development. The topics included in this paper are as 
follows: the groundwork, consortium organization, maximizing survey participation, ensuring data 
quality, compiling data analyses, confidentiality of data and recommendations on consortium 
development. 
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The Consortium for Student Retention Data Exchange: 

A Case Study of Consortium Development 

Transforming data into useful information is an important function of institutional research. 
This information enrichment process often requires the analysis of data in the context of 
interinstitutional comparisons. Saupe (1990) identified comparative analysis as a “common 
technique of institutional research.” Harrington and Chen (1995) noted that “The collection, analysis 
and dissemination of institutional data could be traced to peer exchange between Harvard and the 
founding of Yale in 1701 (Cowley, 1960; Dressel, 1971; and Tetlow, 1979).” This common 
technique appears to have been in existence since the beginning of American higher education. 
Recent trends suggest that the number of interinstitutional data-sharing activities has been on the rise 
and the need for comparative data will only increase (Trainer, 1996). 

Data-Exchange Consortia 

As a way to facilitate the increasing need for complicated data exchanges, several consortia 
have been established since the mid-1960s. The approaches for organizing these consortia vary: 
some consortia are peer based; others, issue oriented. Major distinctions between these two 
approaches exist in the criteria required for membership and in the scope of data-exchange activities. 
Peer-based consortia define their membership based on a set of common institutional characteristics. 
Issue-oriented consortia enlist their participants based on a common interest in the same data issue. 
Typically, peer-based consortia facilitate multi-dimensional data exchanges within a pre-selected 
group of similar institutions, while issue-oriented consortia organize data exchanges on one focused 
issue among highly diverse institutions. 

Examples of peer-based consortia include the Southern University Group (SUG) which was 
established in the mid-1960s (www.sair.org/about_sair); the Association of American Universities 
Data Exchange (AAUDE), in 1973; and the Higher Education Data Sharing Consortium (HEDS), in 
1983 (Shaman and Shapiro, 1996). Examples of issue-oriented consortia include the Cooperative 
Institutional Research Program (CIRP) which was established in 1966 
(www.gseis.ucla.edu/heri/heri.html, 2002); Oklahoma State University Faculty Salary Survey by 
Discipline, ini 974 (www.okstate.edu/pbir/IR/FacultySalary.html); and National Cost Study of 
Instructional Costs and Productivity, in 1992 (www.udel.edu/IR/cost, 2002). 




2 



Information Technology and Consortium Development 

A major reason for the recent growth in data-exchange activities is the advancement in 
information technology. The 1970s marked the emergence of information systems on college 
campuses. This development made it possible for institutions to conduct timely data analysis on a 
variety of institutional research topics. At first, the mainframe-based information technology was 
available to a limited number of large research universities. In the computing environment of the 
early 1980s, using punched cards for data exchange was celebrated as “an improvement over the 
questionnaire, in that the participant’s computer is talking directly to that of the coordinator— no 
‘typos’ creep in to impede communication, no human data transcription effort is required, and it is 
not necessary to send data listings to participants for verification.” (Bloom and Montgomery, 1980). 

In just two decades, the medium for data exchange has progressed from paper files, punched 
cards, electronic files, to shared on-line databases. The improvement in networking technology has 
significantly reduced the resources required for data collection, entry, analysis and dissemination. 
Issue-oriented consortium organizers no longer have to limit the number of participants because of 
resource concerns. On the contrary, they can benefit from increased membership in many ways. 
The larger an issue-oriented consortium is, the more likely it can deliver the following advantages: a 
richer data context for comparative analyses, a more diverse pool of institutions for constructing 
meaningful comparison groups and a lower cost for individual members. 

Moreover, the advancement in microcomputer technology in the past two decades has 
extended the capabilities for producing complicated data analyses to even the smallest and the most 
distant campuses. The spread of automated data systems has enabled institutions to compile data on 
a variety of issues regularly. Today, most institutions routinely participate in data-exchange 
activities. Lack of data is no longer an issue that deters institutions from participating in a data- 
exchange consortium. These recent developments have contributed to a much more favorable 
environment for conducting interinstitutional data exchanges, especially for organizing large, issue- 
oriented consortia. 

As a product of this new information age, the Consortium for Student Retention Data 
Exchange (CSRDE) was established in 1994. This issue-oriented consortium began with a diverse 
membership of 163 institutions. By 2001, the membership had increased to about 420 colleges and 
universities. In its eight-year history. The CSRDE has accumulated a valuable collection of 



experiences in consortium organization and in data-exchange coordination. The purpose of this 
paper is to document these experiences and to present the CSRDE as a case study for future 
consortium development. 

THE GROUND WORK FOR THE CSRDE 
Background 

The CSRDE Retention Survey evolved from a survey instrument originally designed for 
collecting the retention and graduation rates of first-time freshmen in the Big Eight, the Big Ten and 
the SUG institutions. Since the early 1980s, the University of Oklahoma (OU) has participated in 
two peer-based data-exchange consortia: the Big Eight Data Exchange (BEDE, changed to the Big 
Twelve Data Exchange in 1998) and the SUG. Because of geographical proximity, data-exchange 
activities in the BEDE often include the Big Ten public universities. 

The BEDE and the SUG consortia work in very similar ways. The members meet at least 
once a year in the fall to exchange data and to share information on issues and expertise related to 
institutional research (IR). There are two ways in which data sharing takes place within each group: 
(1) ad hoc exchanges of various one-time data inquiries and (2) a defined package for regular and in- 
depth data sharing on selected data items. In the SUG data exchange, each data item has a 
designated coordinator. It is the responsibility of each coordinator to conduct survey activities and to 
conclude the process by distributing a comparative report to the participants. This process has 
yielded many useful and quality comparative analyses: Teaching Load Data Exchange, 
Administrative Salary Survey, Faculty Salary Survey, etc. This SUG data-exchange model was later 
adopted for organizing the CSRDE. 

In 1989, as a part of the university planning at OU, student retention was identified as a 
critical area for improvement. When the university began its initiatives to increase student retention 
and graduation rates, the IR director anticipated that routinely produced retention reports, including 
data from the comparison groups, would soon become an integral part of the process. In the absence 
of a regularly conducted student retention data exchange, the OU IR director proposed a new 
retention data exchange at the annual BEDE meeting. The proposal was accepted by the BEDE 
representatives; and the proposer was asked to design the required survey instruments and to 
coordinate the BEDE retention survey. 

Coincidentally, the SUG institutions also decided to include student retention as a SUG data- 
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exchange item at the annual SUG meeting two weeks later. To avoid duplication of efforts in 
retention survey design, the OU IR director, also a SUG member, became a convenient choice for 
coordinating the SUG retention survey. In a short time, the retention data exchange had assembled 
more than 30 institutions as its base for the first survey. 

It is worth noting that among individual campuses, issues often emerge concurrently in 
response to significant national trends. In this case, several events in the 1970s and 1980s led to the 
common need for comparative retention data at many colleges and universities. First, the economic 
recession of the mid- 1970s prompted greater demand for accountability by state and federal agencies 
(Peterson, 1986). Consequently, assessing student outcomes became increasingly important at 
colleges and universities. Second, enrollment projections forecasted a declining college-aged 
population for the late 1980s and 1990s. Increasing student retention rates became a strategy 
adopted by many campuses to soften the impact of the projected enrollment decreases. Third, 
minority college population increased at an accelerating rate throughout the 1980s with a total 
increase of more than 50 percent in one decade. Retention rates were significantly lower for this 
fast-growing minority population (Smith, 1995). Fourth, more than half of the BEDE, the Big Ten 
and the SUG universities began routinely reporting internal student-tracking data in the mid-1980s. 
By 1989, they were looking for a comparative context for interpreting the retention and graduation 
rates. 

The Original Survey 

The retention survey design began soon after the BEDE and the SUG meetings. The 
preliminary survey was then distributed to the consortia members for their review. This pre-survey 
review served an important function in ensuring a successful data exchange. In examining the 
preliminary survey instrument, consortium members could determine first-hand how data would be 
gathered and what data items would be required. The process also generated valuable comments and 
suggestions from the BEDE and the SUG representatives. They helped define survey data items, 
organize survey procedures and thereby reduced potential difficulties and problems. After several 
revisions, the survey instrument was finalized in late January of 1990. 

The inaugural survey was distributed to 43 universities in the Big Eight, the Big Ten public 
and the SUG universities. Thirty-nine, or 91 percent, of the 43 universities responded to the survey. 
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Each year, separate reports were compiled as a result of this survey: the Big Eight Retention Study, 
the Big Ten Retention Study and the SUG Retention Study. The survey time line was simple: (1) 
survey distribution by March, (2) survey due in July, and (3) dissemination of comparative studies in 
August. The strategy for formulating comparison groups for these peer-based consortia was pre- 
defined. It simply followed the membership structure of each consortium. 

A Sustainable Survey Base 

Beside the consistently high response rates of the BEDE, the Big Ten and the SUG retention 
surveys, there were other indications of a wider interest in comparative retention data. In 1994, the 
AIR Forum evaluation results showed that 58 percent of the respondents ranked student persistence 
and retention as highly important to their professional role (Lindquist, 1999). Two years after the 
first comparative retention reports were published, the survey coordinator was invited to present 
information about the retention data exchange at several regional and national conferences. They 
included a Committee on Institutional Cooperation Conference (consisting of the Big Ten 
universities and the University of Chicago) and workshops sponsored by the National Science 
Foundation (NSF). From 1993 to 1995, similar studies were conducted for more than 70 institutions 
that participated in two NSF educational programs nation-wide. 

By 1 993 , the survey instruments had been introduced to more than 1 00 four-year colleges and 
universities in six separate groups. These institutions represented a wide range of characteristics in 
mission, geographical location, admission standards and size. Despite the extensive scope of data 
collection required by this survey, the response rate was encouraging. “Sixty-seven of those 
institutions provided full responses to all racial subgroups surveyed.” (Smith, 1995) By 1994, many 
other institutions had also expressed their desire to participate in the retention data exchange. 

These events and several other conditions gave sufficient reasons for believing that a 
retention data exchange consortium was sustainable. First, the survey instrument demonstrated its 
applicability beyond those in the BEDE, the Big Ten and the SUG. Survey experiences with three 
other diverse groups of more than 70 institutions indicated that potentially 1,400 colleges and 
universities could benefit from the survey. Second, the survey process and logistics did not create 
significant added burdens to either the participants or the host institution. Third, the comparative data 
proved necessary and useful for the host institutions and for others in the higher education 
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community. Fourth, the survey base persisted and was likely to grow. The survey base might be 
further enhanced by the Student Right-to-Know and Campus Security Act requires institutions 

to make available their graduation rates. Fifth, in the late 1980s and early 1990s, the strategic plan at 
OU actively encouraged external professional activities that could enhance the university’s national 
visibility, reputation and/or resources. As a part of the strategic plan, national outreach became a 
part of the IR office mission. These events culminated in the decision to establish the CSRDE in 
August 1994. 

ORGANIZING THE CSRDE 
Membership Recruitment 

Organizing the CSRDE began with membership recruitment. The prospective members were 
identified as four-year baccalaureate degree-granting colleges and universities. A recruiting package 
was sent to the president of each institution in late August. The package consisted of an invitation 
letter, a sample survey form, a calendar of activities and a registration form. The recruiting letter 
outlined the purpose, activities and responsibilities of the consortium; it also explained membership 
obligations and privileges. This simple letter presented important contractual information to 
prospective members for consideration. 

The recruiting process recurred each of the subsequent seven years. Beginning with the 
second year, an executive summary of the previous year’s CSRDE Retention Report was added to 
the package. In 1998, recruiting information was also posted on the web site, 
www.occe.ou.edu/csrde. It has proved to be an effective and convenient tool for communicating 
with the general public, the CSRDE members, and especially the prospective members. 
Consortium Activities and Time Line 

The CSRDE offers activities on two separate tracks. The core survey track relates directly to 
the retention data exchange and the supplemental track pertains to the annual CSRDE effective 
retention program award. The supplemental track solicits from individual campuses the case studies 
that document how institutional and student characteristics have affected the retention and 
graduation rates of their students. These presentations added perspectives and dimensions to the data 
exchange. The supplemental track also contributed to more frequent and active communications 
within the consortium. The combined activities of these two tracks were scheduled evenly over the 
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course of a year. In doing so, there were periodical, if not monthly, contacts between the survey 
organizer and the consortium members. 

Special attention was given to the timing of survey distribution and report dissemination. The 
timing of survey distribution and data collection took into account the IPEDS reporting schedule, 
workload at the IR offices and scheduled holidays and vacations. Where possible, survey data 
definitions adhere to the IPEDS standards. Each year, the survey was distributed in mid-December 
or in early January. By then, the busy fall reporting season was over and institutional enrollment and 
retention files were available at most institutions. 

MAXIMIZING PARTICIPATION 

The principal rule for disseminating reports is timeliness. As a primary condition for joining 
a data-exchange consortium, participants expect timely and accurate information from the group 
(Teeter and Brinkman, 1992; Trainer, 1996; Sapp, 1996). Therefore, a consortium’s existence often 
hinges on its ability to deliver accurate information before it can be obtained from other public 
sources. In the case of the CSRDE, the tum-around time between the last survey response and the 
finalized annual report was usually shorter than 30 days. Secondary considerations for report 
completion also existed. For example, the date for finishing the CSRDE report was usually targeted 
at about two weeks before the annual AIR Forum so that the reports could be distributed to the 
participants before the conference and the Executive Summary brochures could be printed in time for 
distribution at the conference site. 

Deadlines 

Trainer (1996) observed that “The key to success for a data exchange is the cooperation of its 
members.” From a survey coordinator’s perspective, perhaps the most challenging task in a data 
exchange is finding ways to maximize this cooperation. The survey response rate was 75 percent in 
the early years of the CSRDE and later improved to about 85 percent. While the majority of the 
institutions consistently completed their surveys before the due date, it was common for others to 
submit their responses after the deadline had passed. In scheduling survey activities, it is important 
to anticipate some delays. To maximize participation, a useful practice is to remind survey 
participants of deadlines, multiple times if necessary. When working with the CSRDE group, the 
survey coordinator usually sent three reminders: first an e-mail reminder two weeks before the 
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survey due date, followed by another one shortly past the due date, and a final prompt by phone, two 
weeks later. This was time-consuming but effective. 

Survey Instrument 

Keeping the survey instrument manageable for the participants is a key step in achieving a 
successful response rate. Special efforts were made to keep the survey size minimal. The data 
elements were reviewed carefully and only those essential items were included. Data items that may 
not be available at some institutions were included in the survey as “optional” items so as to 
encourage participation. It is important to note that the CSRDE survey experiences showed that over 
time both the quality and the quantity of survey responses improved. Eventually, the “optional” 
items, once considered novel, would become routinely available at most survey institutions. Overall, 
the survey response rate also improved, from 75 percent in 1995 to 85 percent in 2001. 

Flexible Data Submission Formats 

The CSRDE survey participants represented a wide range of institutional diversity, from 
public research universities with more than 50,000 students to private baccalaureate colleges with 
fewer than 800 students. When coordinating data-exchange activities in a large and diverse 
consortium, it is necessary to allow flexibility in survey response formats so that institutions could 
participate more easily. In the earlier years of the CSRDE, the format for data exchange was simple. 
One set of hardcopy survey instruments was used for all participants. Increases in membership 
diversity and the advancement in networking technology brought about simpler and more convenient 
formats for several groups within the CSRDE. 

Adapting to a different data-sharing format can benefit both the participating and the host 
institutions. In 1995, the California State University System (CSU) of 22 campuses joined the 
CSRDE. To submit data for all these campuses, one single electronic ASCII file from the existing 
CSU reporting system was used to conveniently transmit the required data to the CSRDE computer. 
In 1999, AAUDE members participated in the survey using their own retention data exchange 
format. Similar arrangements were made for several other groups: the University System of 
Georgia, the University of North Carolina System, and the Oklahoma State Regents for Higher 
Education. In 1999, many CSRDE members indicated a preference to submit survey data in an 
electronic Excel file format. In response, an Excel survey form was placed in the CSRDE web site. 



In 2001, about one third of the institutions used the Excel survey format and the file was submitted 
as an e-mail attachment. The next evolutionary development would be on-line web-based data 
submission. At the same time, for some institutions, paper processing may remain their preference. 

ENSURING DATA QUALITY 
A Sound Survey Instrument 

Ensuring the quality of survey data begins with a sound survey instrument. The survey data 
elements should be clearly defined and the survey contents well researched. As stated earlier, the 
CSRDE retention survey instrument benefited from the pre-survey review by the BEDE and the 
SUG representatives. Later, the survey was also used for data collection in more than 100 
institutions. As a result, potential problems were substantially reduced. Other measures were 
included to help prevent potential errors: a checklist with criteria for editing common errors and 
built-in editing functions to flag errors in the electronic spreadsheet so that corrections can be made. 
A Vigorous Editing Process 

To ensure the quality of data, it is vitally important that a vigorous editing process be 
pursued. A stringent editing process was put in place to correct as many errors as possible. The 
cause for these errors ranged from simple mechanical or data entry errors, to illogical data, 
inconsistency problems and misstated facts. Several editing steps were carried out to ensure data 
accuracy. 

First step: preliminary edit. When each survey was received, the responses were visually 
inspected for completeness and general accuracy. This preliminary edit allows timely corrections by 
the survey respondent. 

Second step: edit clerical or data-entry errors. This process was separately performed by at 
least three individuals. Each would check through the full set of materials once. For survey data that 
were submitted electronically, this step would not be necessary. 

Third step: document the status of data reported. Each institution’s survey responses were 
reviewed thoroughly. Missing data and any differences from the standard definitions were 
documented (Figure 1). This documentation was included in the annual report as an appendix. 
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Figure 1: Sample page of survey documentation with Illustration data. 

Documentation of 2000-01 CSRDE Retention Survey Responses 



*(6 

Q 

0) 

& 

:3 

CO 

(D 



<0 




to 


0) 




0) 


>> 




>* 


lo 




lo 








o 




o 


k*— 




k*— 


T3 




T3 


0) 




0) 








s 






c 




c 


0) 




0) 


o 




o 


0) 




Q> 




0> 

c 




h- 


h- 


< 


o 


< 


CO 


2 


CO 



JU 

(D 

> 

(D 

'o 

C 

(/) 

0) 



o 

c 



T3 

0) 

c 

0) 

(/) 

0) 

Q. 

0) 

O 

TD 

C 

3 



o 

s: 

o 

o 

in 

O) 

CO 

O) 

O) 



-D 

0) -o 
ro 0) 

E ? 



■O 

(D 



</) 



O 



<u .E 

0) o 
c 

CO 

T3 *5 

§■ ^ 

^ 3 

O O 
' *r~ 

o iz 
o to 
^ p 

CN ^ 

o 
E 
E 

(0 3 

Q CO 



T3 
0) 

T3 
_3 
o CC) 

X E 
0) .E 

(/) (/) 

C 0) 

•2 -c 
c o 

T3 ^ 

CJ) 

(0 O) 

'o 

OJ (U 
Cl jo 



(0 



CJ) 

CJ) 

CJ) 



3 

O 

CL 

O 



■o 

0) 

T3 

_3 

O 

c 



c 

0) 

T3 

3 

(/) 

0) 

O) 

c 

CO 

s: 

o 

X 

LU 



in 

CJ) 

C3) 



c 

CO 

O) 

0) 

Xi 



c 

0) 

T3 

3 

-4—' 

(/) 

c 

o 

‘> 

I— 

0) 

$ 

o 



_Q 

JO 

to 

> 

CO 

'o 

c 

(/) 

CO 

0) 

>» 

s: o) 

p i 

T3 Q- 
C « 

" -s 



(/) 

Q. 

3 

O 

k_ 

O) 

Xi 

3 

(/) 

0) 

O 

CO 

0) 

> 

o 

0) 

Q. 



CJ) 

CJ) 



T3 

CO 

O 



o 

CD 



CD to 
^ tu 

^ lo 
0) 



P ••;= 



_0) 
X) 



(/) 

3 

V) U) 

c 0) 

0) >- 



CO 

3 

TJ 

2 

O) 



CO 

> 

to 

o 

c 



CO 

T3 

c 

0) 



T3 

C 

0) 

c 

o 



o 

00 

E 

o 



X3 

JO 

to 

> 

CO 

o 

c 

CO 

0) 

>> 



■O 
(U 

o 
a. 

0) 

(/) 
c 

•.= 0) -= p 



c »- 

0) a) 



0) 





o 








c 

o 






-4—' 

c 


0) 

(/) 


c 




c 










o 




0) 


(0 


0) 




0) 








CO 


x: 




■U 


X2 


■U 




E 








3 

_C 


o 

o 








0) 


0) 




0) 


0) 


0) 




CO 


0> 




o 




c 




c 


c 


c 


C 


CJ) 


c 


£I 


xz 


o 


o 


C 


o 


o 


o 


o 


C3) 


o 


o 


o 


o 


2 


LU 


2 


2 


2 


O 


T — 


2 


2 


o 


2 



0) 

E 

CO 



O 'O 



CO 
3 
C 

<7 <i> o 

CN Q O 



c 



c q: 

o 

o 



(/) 

0) 



o 

6 

o 

o 

CM 



•Q 
q> (D 



*s 

5“ 



t: 

o 

a 

0) 

q: 



c: .fc 0) 



0) 

a 



t: 

CO 



0. 



o 



to 

§ 

o 

CO 



42 ^ 

CO 

I 

a -^2 

o> g 

oc ■§ 



Cl 

JO 

*10 

Q 

D) 

V) 

.*0 

§ 



o o 

of of a: 



(0 (0 
T3 T3 



t/) t/) 

(U (U 



O 

of < < 



(0 

T3 



(/) 

(U 



O) 

o S 

a: 



o 

of 



q: 



X X X X X 



XXXIX 



X I I I I I X I X X 



I I I I I 



I I I X I 



I X X X X XXX* I 



< 


CD 


o 


Q 


LU 


LL 


o 


X 


— 


“) 








2 


O 


X 


O 


q: 


CO 


h- 


3 


> 


5 


X 


> 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


0 


0 


0 


0 


0 


0 


0 


3 


3 


3 


3 


3 


‘-4-> 

3 


3 


3 


3 


3 


‘-4-> 

3 


•4-f 

3 


3 


3 


3 


3 


3 


'3 


3 


'3 


'3 


'3 


*3 


"3 


"3 


-4— » 




-4— < 














-4— < 
































*>4-J 

(/) 


(/) 


(/) 


‘•4-' 

(/) 


‘•4-' 

(/) 


(/) 


(/) 


•4-' 

(/) 


•4^ 

(/) 


•4—' 

(/) 


(/) 


•4— ' 
(/) 


•4—' 

(/) 


•4—' 

tn 


•4—' 

(/) 


•4—' 

(/) 


-4— < 
(/) 


'4— < 
(/) 


-4—' 

(/) 


-4-^ 

(/) 


-4-^ 

(/) 


-4-^ 

(/) 


-4-^ 

(/) 


T) 


T) 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 


c 




13 



(1) SAT data recentered for ail years 

(2) Missing Data: R-data missing for race subgroups; G-data missing for gender subgroups 



Fourth step: edit illogical errors, inconsistency problems and, sometimes, misstated facts by 
an automated editing process. An example of each type of data errors is provided as follows. (1) 
Illogical data: four-year graduation rate was greater than six-year graduation rate for the same cohort. 
(2) Inconsistency problems: the sum of males and females did not match the total. (3) Misstated 
facts: recentered SAT scores were reported for the years where old SAT scores were required. Type 
three errors often required a double check by an analyst for confirmation. They could not always be 
detected directly by the computer programs alone. However, it was helpful for the programs to flag 
potential errors, followed by a data analyst’s investigation. 

Fifth step: a final review of the data by a seasoned analyst. This step was carried out as a 
final edit to correct some errors that would require historical knowledge and background information 
about the project. The experiences and insights acquired from working with a project could not be 
easily built into an automated editing process. 

Documenting Survey Data 

“Regardless of the apparent usefulness of an exchange, one must remember that a perfect fit 
in definitions or in collections procedures among the participants is extremely difficult to attain, and 
as a consequence, some lack of synchronization in data and comparison should be expected.” 
(Bloom and Montgomery, 1980) Documentation helps clarify these data differences. As a part of 
the documentation, missing data elements could also be noted (Figure 1). Thorough documentation 
contributes to a better appreciation of reporting standards and data quality. Many survey participants 
have come to regard the documentation of the CSRDE survey responses as a report on their 
performance in the data-sharing process. Even though this was not the original intent, the 
documentation did contribute to improved survey responses over the years. 

Another aspect of data documentation is the data-editing process. When survey responses 
did not meet reporting requirements, they were excluded from the final report. Sometimes, only 
minor errors existed and editing could solve the data problems. Because of the short tum-around 
time, most of the editing activities occurred in-house. Instead of notifying individual institutions 
when their data required editing, documentation in the final report was a more practical way to keep 
institutions informed. The following is an example of the editing notes: 

“Because of the static nature of retention and graduation rates, infrequent missing 

data cells for an institution were replaced with the corresponding column averages 
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for that institution. This practice is necessary to ensure a more consistent 
comparison.” (Smith, 2001) 

Once the report was finished, the edited database became the official data source for the entire year. 
The database was used to compile peer comparison reports when they were requested; it was also 
used to generate new survey forms for the next data collection cycle. Multiple opportunities existed 
for survey participants to review their data as stored in the CSRDE database. 

Data Analysis 

In the CSRDE retention survey, data was shared at the aggregated level by individual 
institutions. Retention data was collected by race and by gender. To assist in the process of peer 
identification, data pertaining to institutional and student characteristics were also collected as 
supplements. The selection of characteristics data depended on three factors: (1) data availability, 
(2) relevance to student retention, and (3) applicability for peer identification. Institutional size, 
student age, average admission test scores, high school ranking, first semester GPA were a few 
examples of institutional and student characteristics data. “Whenever feasible, exchange activity 
should build upon work the institutions are already doing.” (Trainer, 1 996) Some supplemental data 
was derived from published reports to reduce the survey burden and to ensure consistency. For 
example, the data on institutional classification came from the Carnegie Classification of Institutions 
of Higher Education. 

The CSRDE reports consistently indicated that institutional selectivity was the most 
significant factor in comparing institutional retention and graduation rates. Therefore, in the context 
of student retention comparison, institutional selectivity was the key criterion used for formulating 
meaningful comparison groups. Other significant factors included student race and gender, percent of 
part-time students, and public vs. private institutions. The annual retention report was divided into 
four sections of varying selectivity; highly selective, selective, moderately selective and less 
selective. Summary tables also displayed comparisons by race, gender, and other characteristics 
cited earlier. Two appendices, which provided extensive data about individual institutions, were 
included for reference purposes (Figure 2). These detailed data were useful for identifying 
institutional comparison peers. 

In addition to the armual report, a six-page Executive Summary was produced for extensive 



Figure 2: Illustrations for (1) display of characteristics data and (2) formats for displaying confidentiality requirement. 
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distribution in the higher education community. As a special privilege, survey participants could 
request a customized comparison report by submitting a list of self-identified peers. Less than 40 
percent of the institutions requested customized reports each year. The annual report itself appeared 
to have been satisfactory for most of the consortium members. For those who requested customized 
reports, the selections reflected an interesting array of comparison groups, including selectivity 
peers, predetermined groups, in-state or regional competitors, and aspiration institutions (T eeter and 
Brinkman, 1992). 

Data Confidentiality 

The practice of honoring an institution’s need for data confidentiality encourages data- 
exchange members to entrust their data with a consortium. This trust requires the cooperation of all 
consortium members. In recent years, only a handful of the institutions required data confidentiality 
—six of 344 institutions in 2001 . The CSRDE data exchange is virtually an open process within the 
group. However, extra care was taken to prevent the potential misuse of data. A request for 
confidentiality was posted on each page where institutions were identifiable (Figure 2). Within a 
data-exchange group, it is essential to foster the culture of good stewardship for data. In the 
environment of trust, institutions can feel confident about sharing information with others. 

Review the Consortium Activities 

Improving institutional performance requires regular and timely evaluation of activities. 
Three sources of reviews were regularly pursued at the CSRDE. The most frequent source of 
reviews came from the individual participants. In the early years of the CSRDE, post-exchange 
surveys of participants were conducted. Later, because of the frequent contacts between the 
coordinator and the consortium members, this process was replaced by informal discussions. 
Working discussions with members by phone, via e-mail or at meetings brought about many good 
ideas for the CSRDE activities as well as improvements in the annual report format. This is one of 
the reasons why the CSRDE report was usually distributed in advance of the AIR conference. 
Conversations at the conference about the annual report were helpful exercises. 

An official source of evaluation came from the external reviewers. Each year, two 
distinguished members in the higher education community were invited to review the CSRDE 
activities. Depending on the special need of a given year, the reviewers’ expertise areas could vary. 
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from institutional research, higher education administration to information technology. Beginning 
with 1999, an advisory group of six seasoned institutional researchers was added to the review 
process. These external reviewers provided expert assessments and opinions about the CSRDE 
activities from different points of view. Their participation also strengthened the visibility and the 
prestige of the CSRDE. 

The final source of review was the host institution itself. A log was kept in the office to 
record events that require improvements for implementation in the following year. The areas subject 
to this continuing review included: the reporting format, the logistics of a specific event, survey 
procedures, web design, computing software for data editing, etc. On-going learning and research in 
national databases, advancement of technology, student retention research and other current events 
were a necessity. The newly acquired knowledge was crucial in updating the standards for 
evaluating the consortium’s performances. 

Changes at the CSRDE 

In its eight-year history, several changes occurred at the CSRDE. The consortium 
membership increased from 163 colleges and universities in 1994 to more than 420 in late 2001. 
Included in the membership was at least a dozen data-exchange consortia or state higher education 
systems. The media for data exchange moved from strictly paper to more convenient ASCII file 
transfers or other e-mail attachments. The survey instruments became more sophisticated and the 
reports, more comprehensive. Gradually, the survey environment shifted from a closed exchange 
towards an open data sharing within the CSRDE. Unlike the early years, institution names were no 
longer masked with unidentifiable numbers. In 1999, the CSRDE began expanding services to its 
members by leveraging external resources. A three-year NSF research grant was awarded for a data 
exchange in the retention and graduation rates of science, technology, engineering and mathematics 
(STEM) majors. This sponsored funding made it possible for institutions to exchange retention data 
on their STEM students. When needed, institutions could receive a small grant to supplement the 
cost of survey participation. These many changes could not have happened without the extraordinary 
collaboration of more than 400 institutions, in particular, the CSRDE charter members. 

Summary 

Since its establishment, the CSRDE has accumulated many remarkable experiences and 
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lessons. First, the development in information technology has removed barriers that once 
discouraged consortium development. Most institutions are now able to produce data on a variety of 
issues regularly and they participate routinely in data-exchange activities. The advancement in 
networking technology has substantially improved the ease in data collection, editing, analysis and 
dissemination. Consequently, the cost associated with data exchanges has become affordable for 
most institutions. These factors will contribute to a more favorable environment for developing 
large, issued-oriented consortia in the future. 

Second, it takes grass-root support from individual institutions to develop and sustain a 
consortium. An easier and surer way to start a consortium is by working with a small number of 
institutions in the beginning. The CSRDE began with a dedicated group of professionals in the 
BEDE, the Big Ten and the SUG institutions. They had similar insights, purpose and commitment 
in developing a mechanism for exchanging retention data. The collaboration in the earlier years of 
the retention data exchange paved the way for a smooth data-exchange procedure and set high 
standards for the quality of data submitted. 

Third, “Make participation in the exchange as easy as possible.” (Trainer, 1996) This is 
especially true in developing issue-oriented consortia with a large and diverse membership. To 
maximize participation, it is important to keep the amount of survey data minimal. In the cases 
where data may not be commonly available, label the requested data elements “optional.” In time, 
more institutions will be able to fully participate. Another way to make it simpler for participation is 
to allow maximum flexibility in the medium and the format for data submission. 

Fourth, “An exchange should deliver a consistent, high quality product or service in a timely 
fashion and in a format that is useful to its members.” (Trainer, 1996) This data-exchange ground 
rule is one of overriding importance. There is no substitute for accuracy and the credibility of a data- 
exchange consortium. As much as possible, measures should be taken to ensure the quality of data: 
beginning with a quality survey instrument, followed by a stringent and vigorous editing process, a 
thorough documentation on survey responses, and a sound data analysis procedure. With the best of 
efforts, small errors can still happen in the annual report. When errors do occur, it is important to 
make the necessary corrections and to notify the data users immediately. 

Finally, it is important for a consortium to foster a culture of good stewardship for data. In an 
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environment of trust, institutions can feel confident in sharing information with others. 

Conclusion 

Relative to the long list of topics that could benefit from further comparative analyses, very 
few issue-oriented consortia exist. In a large issue-oriented consortium, a diverse community can 
come together to examine a common issue of interest. In return, the consortium can benefit its 
members in many ways: a larger context for comparisons, an in-depth and longitudinal investigation 
into an issue, an opportunity for networking, a common depository of expertise, a comprehensive 
database for formulating meaningful comparison groups, and a cost-efficient way to obtain 
comparative information. In the future, these benefits will increase the demand for more issue- 
oriented consortia. Ultimately, these newly organized consortia will contribute to better decision 
making on individual campuses. 
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